Function Call Accuracy Analysis

Total Queries

47

Successful Queries

41

Success Rate

87.2%

Overall Metrics

Agent Name Accuracy: 93.6%

Plugin Name Accuracy: 93.6%

Arguments Accuracy: 87.2%

Overall Accuracy: 87.2%

Agent type Distribution

Overall Accuracy Metrics

Overall Accuracy by Agent Type

Query Details

Query Response Skill Type Intents Functions Agent Name Arguments Overall
turn off washing machine The washing machine has been turned off. [ washingmachine_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:washingmachine_control Arguments:{"operation": "start"} Result:Washing machine stop.
Predicted Function:
Agent:control_device_operation Plugin:washingmachine_control Arguments:{"operation": "stop"} Result:Washing machine stop.
✓ ✗ ✗
start the washing machine The washing machine has been started. [ washingmachine_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:washingmachine_control Arguments:{"operation": "start"} Result:Washing machine start.
Predicted Function:
Agent:control_device_operation Plugin:washingmachine_control Arguments:{"operation": "start"} Result:Washing machine start.
✓ ✓ ✓
pause the washing machine The washing machine has been paused. [ washingmachine_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:washingmachine_control Arguments:{"operation": "pause"} Result:Washing machine pause.
Predicted Function:
Agent:control_device_operation Plugin:washingmachine_control Arguments:{"operation": "pause"} Result:Washing machine pause.
✓ ✓ ✓
resume the washing machine The washing machine has been resumed. [ washingmachine_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:washingmachine_control Arguments:{"operation": "resume"} Result:Washing machine resume.
Predicted Function:
Agent:control_device_operation Plugin:washingmachine_control Arguments:{"operation": "resume"} Result:Washing machine resume.
✓ ✓ ✓
stop the washing machine The washing machine has been stopped. [ washingmachine_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:washingmachine_control Arguments:{"operation": "stop"} Result:Washing machine stop.
Predicted Function:
Agent:control_device_operation Plugin:washingmachine_control Arguments:{"operation": "stop"} Result:Washing machine stop.
✓ ✓ ✓
set the washing machine to quick wash mode The washing machine is now set to quick wash mode. [ washingmachine_control ] Single Intent
Expected Function:
Agent:set_mode Plugin:washingmachine_control Arguments:{"mode": "quick wash"} Result:Washing machine set to quick wash mode.
Predicted Function:
Agent:set_mode Plugin:washingmachine_control Arguments:{"mode": "quick wash"} Result:Washing machine set to quick wash mode.
✓ ✓ ✓
set the washing machine to delicate mode The washing machine is now set to delicate mode. [ washingmachine_control ] Single Intent
Expected Function:
Agent:set_mode Plugin:washingmachine_control Arguments:{"mode": "delicate"} Result:Washing machine set to delicate mode.
Predicted Function:
Agent:set_mode Plugin:washingmachine_control Arguments:{"mode": "delicate"} Result:Washing machine set to delicate mode.
✓ ✓ ✓
set the washing machine to heavy load mode The washing machine is now set to heavy load mode. [ washingmachine_control ] Single Intent
Expected Function:
Agent:set_mode Plugin:washingmachine_control Arguments:{"mode": "heavy load"} Result:Washing machine set to heavy load mode.
Predicted Function:
Agent:set_mode Plugin:washingmachine_control Arguments:{"mode": "heavy load"} Result:Washing machine set to heavy load mode.
✓ ✓ ✓
set the washing machine timer to 30 minutes The washing machine timer has been set to30 minutes. [ washingmachine_control ] Single Intent
Expected Function:
Agent:set_timer Plugin:washingmachine_control Arguments:{"time": "30 minutes"} Result:Washing machine timer set to 30 minutes.
Predicted Function:
Agent:set_timer Plugin:washingmachine_control Arguments:{"time": "30 minutes"} Result:Washing machine timer set to 30 minutes.
✓ ✓ ✓
set the washing machine timer to 1 hour The washing machine timer has been set to1 hour. [ washingmachine_control ] Single Intent
Expected Function:
Agent:set_timer Plugin:washingmachine_control Arguments:{"time": "1 hour"} Result:Washing machine timer set to 1 hour.
Predicted Function:
Agent:set_timer Plugin:washingmachine_control Arguments:{"time": "1 hour"} Result:Washing machine timer set to 1 hour.
✓ ✓ ✓
turn on the TV and increase the volume The TV has been turned on, and the volume has been increased. [ tv_control , tv_control ] Multi Intent
Expected Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Predicted Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Expected Function:
Agent:adjust_volume Plugin:tv_control Arguments:{"operation": "decrease"} Result:TV volume increased.
Predicted Function:
Agent:adjust_volume Plugin:tv_control Arguments:{"operation": "increase"} Result:TV volume increased.
✓ ✗ ✗
turn on the TV and decrease the volume The TV has been turned on, and the volume has been decreased. [ tv_control , tv_control ] Multi Intent
Expected Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Predicted Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Expected Function:
Agent:adjust_volume Plugin:tv_control Arguments:{"operation": "decrease"} Result:TV volume decreased.
Predicted Function:
Agent:adjust_volume Plugin:tv_control Arguments:{"operation": "decrease"} Result:TV volume decreased.
✓ ✓ ✓
turn on the TV and mute it The TV has been turned on. Unfortunately, muting the TV is currently not available. However, I can decrease the volume if you'd like! [ tv_control ] Multi Intent
Expected Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Predicted Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
✗ ✗ ✗
turn on the TV and set the volume to 15 Setting the TV to a specific volume level is not directly supported; I can only increase or decrease the volume. Would you like me to assist in adjusting the volume incrementally? [ tv_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Predicted Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
✓ ✓ ✓
turn on the TV and switch to HDMI 1 The TV has been turned on, and the input source has been switched to HDMI1. [ tv_control , tv_control ] Multi Intent
Expected Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Predicted Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Expected Function:
Agent:switch_input_source Plugin:tv_control Arguments:{"input_source": "HDMI 1"} Result:TV input switched to HDMI 1.
Predicted Function:
Agent:switch_input_source Plugin:tv_control Arguments:{"input_source": "HDMI 1"} Result:TV input switched to HDMI 1.
✓ ✓ ✓
turn on the TV and open Netflix The TV has been turned on, and Netflix is now open. [ tv_control , tv_control ] Multi Intent
Expected Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Predicted Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Expected Function:
Agent:open_application Plugin:tv_control Arguments:{"app_name": "Netflix"} Result:Netflix opened on the TV.
Predicted Function:
Agent:open_application Plugin:tv_control Arguments:{"app_name": "Netflix"} Result:Netflix opened on the TV.
✓ ✓ ✓
turn on the TV and increase the brightness The TV can be turned on, but brightness adjustments need a specific level set (e.g.,50%,70%). Do you want me to turn on the TV and set a particular brightness level? [ tv_control ] Multi Intent
Expected Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Predicted Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
✗ ✗ ✗
turn on the TV and set the channel to 101 The TV has been turned on, and the channel is set to101. [ tv_control , tv_control ] Multi Intent
Expected Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Predicted Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Expected Function:
Agent:set_channel Plugin:tv_control Arguments:{"channel": "101"} Result:TV channel set to 101.
Predicted Function:
Agent:set_channel Plugin:tv_control Arguments:{"channel": "101"} Result:TV channel set to 101.
✓ ✓ ✓
turn on the TV and enable subtitles The TV has been turned on, and subtitles have been enabled. [ tv_control , tv_control ] Multi Intent
Expected Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Predicted Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Expected Function:
Agent:enable_subtitles Plugin:tv_control Arguments:{} Result:Subtitles enabled on the TV.
Predicted Function:
Agent:enable_subtitles Plugin:tv_control Arguments:null Result:Subtitles enabled on the TV.
✓ ✗ ✗
turn on the TV and switch to HDMI 2 The TV has been turned on, and the input source has been switched to HDMI2. [ tv_control , tv_control ] Multi Intent
Expected Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Predicted Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Expected Function:
Agent:switch_input_source Plugin:tv_control Arguments:{"input_source": "HDMI 2"} Result:TV input switched to HDMI 2.
Predicted Function:
Agent:switch_input_source Plugin:tv_control Arguments:{"input_source": "HDMI 2"} Result:TV input switched to HDMI 2.
✓ ✓ ✓
turn on the TV and open YouTube The TV has been turned on, and YouTube is now open. [ tv_control , tv_control ] Multi Intent
Expected Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Predicted Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "on"} Result:TV turned on.
Expected Function:
Agent:open_application Plugin:tv_control Arguments:{"app_name": "YouTube"} Result:YouTube opened on the TV.
Predicted Function:
Agent:open_application Plugin:tv_control Arguments:{"app_name": "YouTube"} Result:YouTube opened on the TV.
✓ ✓ ✓
turn off the TV The TV has been turned off. [ tv_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "off"} Result:TV turned off.
Predicted Function:
Agent:control_device_operation Plugin:tv_control Arguments:{"operation": "off"} Result:TV turned off.
✓ ✓ ✓
increase the TV volume The TV volume has been increased. [ tv_control ] Single Intent
Expected Function:
Agent:adjust_volume Plugin:tv_control Arguments:{"operation": "increase"} Result:TV volume increased.
Predicted Function:
Agent:adjust_volume Plugin:tv_control Arguments:{"operation": "increase"} Result:TV volume increased.
✓ ✓ ✓
decrease the TV volume The TV volume has been decreased. [ tv_control ] Single Intent
Expected Function:
Agent:adjust_volume Plugin:tv_control Arguments:{"operation": "decrease"} Result:TV volume decreased.
Predicted Function:
Agent:adjust_volume Plugin:tv_control Arguments:{"operation": "decrease"} Result:TV volume decreased.
✓ ✓ ✓
set the air conditioner to 24 degrees The air conditioner temperature has been set to24 degrees. [ ac_control ] Single Intent
Expected Function:
Agent:set_temperature Plugin:ac_control Arguments:{"temperature": "24"} Result:Air conditioner temperature set to 24 degrees.
Predicted Function:
Agent:set_temperature Plugin:ac_control Arguments:{"temperature": "24"} Result:Air conditioner temperature set to 24 degrees.
✓ ✓ ✓
turn off the air conditioner The air conditioner has been turned off. [ ac_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:ac_control Arguments:{"operation": "off"} Result:Air conditioner turned off.
Predicted Function:
Agent:control_device_operation Plugin:ac_control Arguments:{"operation": "off"} Result:Air conditioner turned off.
✓ ✓ ✓
set the air conditioner to cool mode The air conditioner is now set to cool mode. [ ac_control ] Single Intent
Expected Function:
Agent:set_mode Plugin:ac_control Arguments:{"mode": "cool"} Result:Air conditioner set to cool mode.
Predicted Function:
Agent:set_mode Plugin:ac_control Arguments:{"mode": "cool"} Result:Air conditioner set to cool mode.
✓ ✓ ✓
increase the air conditioner temperature to 26 degrees The air conditioner temperature has been increased to26 degrees. [ ac_control ] Single Intent
Expected Function:
Agent:set_temperature Plugin:ac_control Arguments:{"temperature": "26"} Result:Air conditioner temperature set to 26 degrees.
Predicted Function:
Agent:set_temperature Plugin:ac_control Arguments:{"temperature": "26"} Result:Air conditioner temperature set to 26 degrees.
✓ ✓ ✓
decrease the air conditioner temperature to 22 degrees The air conditioner temperature has been decreased to22 degrees. [ ac_control ] Single Intent
Expected Function:
Agent:set_temperature Plugin:ac_control Arguments:{"temperature": "22"} Result:Air conditioner temperature set to 22 degrees.
Predicted Function:
Agent:set_temperature Plugin:ac_control Arguments:{"temperature": "22"} Result:Air conditioner temperature set to 22 degrees.
✓ ✓ ✓
turn on the air conditioner The air conditioner has been turned on. [ ac_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:ac_control Arguments:{"operation": "on"} Result:Air conditioner turned on.
Predicted Function:
Agent:control_device_operation Plugin:ac_control Arguments:{"operation": "on"} Result:Air conditioner turned on.
✓ ✓ ✓
set the air conditioner to fan mode The air conditioner is now set to fan mode. [ ac_control ] Single Intent
Expected Function:
Agent:set_mode Plugin:ac_control Arguments:{"mode": "fan"} Result:Air conditioner set to fan mode.
Predicted Function:
Agent:set_mode Plugin:ac_control Arguments:{"mode": "fan"} Result:Air conditioner set to fan mode.
✓ ✓ ✓
set the air conditioner to dry mode The air conditioner is now set to dry mode. [ ac_control ] Single Intent
Expected Function:
Agent:set_mode Plugin:ac_control Arguments:{"mode": "dry"} Result:Air conditioner set to dry mode.
Predicted Function:
Agent:set_mode Plugin:ac_control Arguments:{"mode": "dry"} Result:Air conditioner set to dry mode.
✓ ✓ ✓
set the air conditioner timer to 2 hours The air conditioner timer has been set to2 hours. [ ac_control ] Single Intent
Expected Function:
Agent:set_timer Plugin:ac_control Arguments:{"time": "2 hours"} Result:Air conditioner timer set to 2 hours.
Predicted Function:
Agent:set_timer Plugin:ac_control Arguments:{"time": "2 hours"} Result:Air conditioner timer set to 2 hours.
✓ ✓ ✓
set the air conditioner timer to 4 hours The air conditioner timer has been set to4 hours. [ ac_control ] Single Intent
Expected Function:
Agent:set_timer Plugin:ac_control Arguments:{"time": "4 hours"} Result:Air conditioner timer set to 4 hours.
Predicted Function:
Agent:set_timer Plugin:ac_control Arguments:{"time": "4 hours"} Result:Air conditioner timer set to 4 hours.
✓ ✓ ✓
start the dishwasher The dishwasher has been started. [ dishwasher_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:dishwasher_control Arguments:{"operation": "start"} Result:Dishwasher startd.
Predicted Function:
Agent:control_device_operation Plugin:dishwasher_control Arguments:{"operation": "start"} Result:Dishwasher startd.
✓ ✓ ✓
stop the dishwasher The dishwasher has been stopped. [ dishwasher_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:dishwasher_control Arguments:{"operation": "stop"} Result:Dishwasher stopd.
Predicted Function:
Agent:control_device_operation Plugin:dishwasher_control Arguments:{"operation": "stop"} Result:Dishwasher stopd.
✓ ✓ ✓
pause the dishwasher The dishwasher has been paused. [ dishwasher_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:dishwasher_control Arguments:{"operation": "pause"} Result:Dishwasher paused.
Predicted Function:
Agent:control_device_operation Plugin:dishwasher_control Arguments:{"operation": "pause"} Result:Dishwasher paused.
✓ ✓ ✓
resume the dishwasher The dishwasher has been resumed. [ dishwasher_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:dishwasher_control Arguments:{"operation": "resume"} Result:Dishwasher resumed.
Predicted Function:
Agent:control_device_operation Plugin:dishwasher_control Arguments:{"operation": "resume"} Result:Dishwasher resumed.
✓ ✓ ✓
set the dishwasher to eco mode The dishwasher is now set to eco mode. [ dishwasher_control ] Single Intent
Expected Function:
Agent:set_mode Plugin:dishwasher_control Arguments:{"mode": "eco"} Result:Dishwasher set to eco mode.
Predicted Function:
Agent:set_mode Plugin:dishwasher_control Arguments:{"mode": "eco"} Result:Dishwasher set to eco mode.
✓ ✓ ✓
set the dishwasher to heavy wash mode The dishwasher is now set to heavy wash mode. [ dishwasher_control ] Single Intent
Expected Function:
Agent:set_mode Plugin:dishwasher_control Arguments:{"mode": "heavy wash"} Result:Dishwasher set to heavy wash mode.
Predicted Function:
Agent:set_mode Plugin:dishwasher_control Arguments:{"mode": "heavy wash"} Result:Dishwasher set to heavy wash mode.
✓ ✓ ✓
set the dishwasher to quick wash mode The dishwasher is now set to quick wash mode. [ dishwasher_control ] Single Intent
Expected Function:
Agent:set_mode Plugin:dishwasher_control Arguments:{"mode": "quick wash"} Result:Dishwasher set to quick wash mode.
Predicted Function:
Agent:set_mode Plugin:dishwasher_control Arguments:{"mode": "quick wash"} Result:Dishwasher set to quick wash mode.
✓ ✓ ✓
set the dishwasher timer to 1 hour The dishwasher timer has been set to1 hour. [ dishwasher_control ] Single Intent
Expected Function:
Agent:set_timer Plugin:dishwasher_control Arguments:{"time": "1 hour"} Result:Dishwasher timer set to 1 hour.
Predicted Function:
Agent:set_timer Plugin:dishwasher_control Arguments:{"time": "1 hour"} Result:Dishwasher timer set to 1 hour.
✓ ✓ ✓
set the dishwasher timer to 2 hours The dishwasher timer has been set to2 hours. [ dishwasher_control ] Single Intent
Expected Function:
Agent:set_timer Plugin:dishwasher_control Arguments:{"time": "2 hours"} Result:Dishwasher timer set to 2 hours.
Predicted Function:
Agent:set_timer Plugin:dishwasher_control Arguments:{"time": "2 hours"} Result:Dishwasher timer set to 2 hours.
✓ ✓ ✓
turn on the dishwasher and start cleaning The dishwasher has been turned on and started cleaning. [ dishwasher_control ] Multi Intent
Expected Function:
Agent:control_device_operation Plugin:dishwasher_control Arguments:{"operation": "start"} Result:Dishwasher startd.
Predicted Function:
Agent:control_device_operation Plugin:dishwasher_control Arguments:{"operation": "start"} Result:Dishwasher startd.
✗ ✗ ✗
turn off the dishwasher The dishwasher has been turned off. [ dishwasher_control ] Single Intent
Expected Function:
Agent:control_device_operation Plugin:dishwasher_control Arguments:{"operation": "stop"} Result:Dishwasher stopd.
Predicted Function:
Agent:control_device_operation Plugin:dishwasher_control Arguments:{"operation": "stop"} Result:Dishwasher stopd.
✓ ✓ ✓
set the dishwasher to sanitize mode The dishwasher is now set to sanitize mode. [ dishwasher_control ] Single Intent
Expected Function:
Agent:set_mode Plugin:dishwasher_control Arguments:{"mode": "sanitize"} Result:Dishwasher set to sanitize mode.
Predicted Function:
Agent:set_mode Plugin:dishwasher_control Arguments:{"mode": "sanitize"} Result:Dishwasher set to sanitize mode.
✓ ✓ ✓
set the dishwasher to rinse only mode The dishwasher is now set to rinse-only mode. [ dishwasher_control ] Single Intent
Expected Function:
Agent:set_mode Plugin:dishwasher_control Arguments:{"mode": "rinse only"} Result:Dishwasher set to rinse only mode.
Predicted Function:
Agent:set_mode Plugin:dishwasher_control Arguments:{"mode": "rinse only"} Result:Dishwasher set to rinse only mode.
✓ ✓ ✓